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Abstract — This paper continues the Wu-Shamai-Verdu pro- 
gram |1| on characterizing the degrees of freedom (DoF) of 
interference channels (ICs) through Renyi information dimen- 
sion. Concretely, we find a general formula for the DoF of vector 
ICs, encompassing multiple-input multiple-output (MIMO) ICs, 
time- and/or frequency-selective ICs, and combinations thereof, 
as well as constant single-antenna ICs considered in |1|. As 
in the case of constant single-antenna ICs, achieving full DoF 
requires the use of singular input distributions. Strikingly, in 
the vector case it suffices to enforce singularity on the joint 
distribution of individual transmit vectors. This can be realized 
through signaling in subspaces of the ambient signal space, which 
is in accordance with the idea of interference alignment, and, 
most importantly, allows the scalar components of the transmit 
vectors to have non-singular distributions. We recover the result 
by Cadambe and Jafar on the non-separability of parallel ICs |2| 
and we show that almost all parallel ICs are separable. Finally, 
our results extend the main finding in 1 1 1 to the complex case. 

1. Introduction 

Sparked by the surprising finding of Cadambe and Jafar 
O stating that K/2 degrees of freedom (DoF) can be re- 
alized in K-user interference channels (ICs) through inter- 
ference alignment, the study of DoF in wireless networks 
has seen significant activity in recent years. The essence 
of interference alignment is to exploit channel variations in 
time/frequency/space to align interference in low-dimensional 
subspaces. This is accomplished through symbol extension and 
a clever choice of the resulting vector-valued transmit signals. 

Following the discovery in [3J it turned out that the basic 
idea of aligning interference to realize DoF can be applied to 
numerous further settings f4\ . Most surprisingly, it was shown 
in Is), H that K/2 DoF can be realized in K-usei constant 
single-antenna ICs, i.e., in complete absence of channel varia- 
tions. The schemes introduced in |5|, |6|, referred to in f5l as 
"real interference alignment", use Diophantine approximation 
or lattice structures to design the transmit signals and rely 
on number-theoretic properties of the channel coefficients to 
achieve K/2 DoF for almost all channels IS). 

Wu et al. im showed recently that (Renyi) information 
dimension fTl] is a suitable tool for characterizing the DoF 
achievable in constant single-antenna ICs in a unified manner. 
Concretely, based on a single-letter characterization of the 
DoF of the constant single-antenna IC, it is shown in HI that 
the real interference alignment schemes proposed in IS), ||6l 
correspond to the use of singular input distributions. Strikingly, 
Wu et al. fll found that full, i.e., K/2 in a K-usev IC, 
DoF can be achieved only by input distributions that have a 
singular component. This is a strong, negative, result as input 
distributions with a singular component are not realizable in 



practice. On the other hand, it is well known that full DoF can 
be realized in ICs with channel variations in time/frequency 
or in constant multiple-antenna ICs, even if the marginals of 
the transmit vectors do not have singular components. It is 
therefore of interest to reconcile these two lines of results. 
We will see that singularity of input distributions plays a 
fundamental role in achieving full DoF in ICs with channel 
variations and/or multiple-antenna users as well, albeit in a 
much more benign form. 

Contributions: We continue the Wu-Shamai-Verdii program 
111 by showing how information dimension can be used 
to systematically characterize the DoF of vector ICs. This 
extension, mentioned in III as an avenue of interesting further 
research, is relevant as "classical" interference alignment relies 
on vector-valued signaling to realize full DoF. The vector IC 
considered in this paper contains, as special cases, the MIMO 
IC, time- and/or frequency-selective ICs, and combinations 
thereof, as well as the constant single-antenna IC studied in 
CI, 13, ID. The Wu-Shamai-Verdii theory builds on a httle 
known, but highly useful result by Guionnet and Shlyakthenko 
|8j Thm. 2.7], stating that the DoF achieved by a particular 
input distribution in a scalar additive noise channel are given 
by the input distribution's information dimension. We present 
an extension of this result to the vector case, formalized 
in Theorem |2] in Section |IVl with the corresponding proof 
provided in |9|. This extension is the key ingredient of our 
main result, a general formula for the DoF in vector ICs. 

The formula is in the spirit of ||T| and allows us to conclude 
that, while input distributions with a singular component are 
still needed to achieve full DoF, it suffices to enforce singu- 
larity on the joint distribution of individual transmit vectors. 
This form of singularity is easy to realize in practice by 
taking, e.g., the transmit vectors to live in lower-dimensional 
subspaces of the ambient signal space, as is, in fact, done 
in interference alignment. We illustrate, by way of selected 
examples, how the main result in this paper allows to treat 
different interference alignment concepts in a unified manner 
Furthermore, we recover the result by Cadambe and Jafar 
on the non-separability of parallel ICs ||2| and we show that 
almost all parallel ICs are separable in terms of DoF. Finally, 
our results apply to complex channel matrices and signal 
vectors, thereby also extending the main result in 1 1 1 to the 
complex case. Most results in this paper are stated without 
proof. Omitted proofs are reported in |9|. 

Notation: Random vectors are represented by uppercase 
letters, deterministic vectors by lowercase letters, in both cases 
using letters from the end of the alphabet. Boldface uppercase 



letters are used to indicate matrices. The diagonal nxn matrix 
with entries ai,...,a„ on its main diagonal is denoted by 
diag(ai, a„). For a; e R, we write [xj for the largest 
integer not greater than x. For fc e N \ {0}, we set {x)k := 
[kx\/k. The last two notation conventions are extended to 
real vectors by applying [-J and {■)k componentwise. For a 
discrete random vector X, we write H{X) for its entropy. 
All logarithms are to the base e. E[ ] denotes the expectation 
operator. 

II. Setup and Definitions 

We consider a memoryless X-user Gaussian vector IC with 
input-output relation 



snr^R 



w,,. 



(1) 



where Xj 



{x,[i]...x,[M])^ e 



pM 



and Y, 



pAf 



is the transmit and receive vector, 
respectively, corresponding to user i, M is the dimension 



of the input/output signal space, ti-ij € j^mxm (jgjjotes 
the channel matrix between transmitter j and receiver i, and 
{Wi)i=i^,,,^K are i.i.d. zero mean Gaussian random vectors 
with identity covariance matrixQ Defining the KM x KM 
matrix 



H 

we can rewrite ^ as 
(Y, 



/snrH 



Hi 



K 
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K.K, 
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Kl 



Kl 



(2) 



The channel matrix H is assumed to be known perfectly at 
aU transmitters and receivers. We impose the average power 
constraint 



M N 



MN 



m— 1 71—1 



,(")r 



[m] ] 1 



of block length 



i = 1,...,K, where x 

pM 



(n) _ 



on codeword matrices 

N transmitted by user 

(xi")[l]...xi")[M])%k . 

This setting encompasses the MIMO IC with M antennas 
at each user and Hij denoting the MIMO channel matrix 
between transmitter j and receiver i. It furthermore con- 
tains time- and/or frequency-selective ICs, where intersym- 
bol/intercarrier interference is transformed into a memoryless 
vector channel through the use of guard periods/bands. When 
all the Hij are diagonal, obtained, e.g., when OFDM is used 

'We show in Section [V-Cl how tlie results in this paper can be extended to 
the complex case. 

^Note that the average is taken across vector symbols. 



in frequency-selective channels ifTol . the model reduces to that 
of M parallel ICs. The constant single-antenna setting consid- 
ered in 1 IJ is covered by the special case M — 1. In contrast to 
the channel model in |[T|, where the action of each subchannel 
is represented through scaling by a single coefficient, in our 
setting each subchannel acts as a linear operator (represented 
by a matrix) on the corresponding transmit vector. 

Let C(H;snr) be the sum-capacit}3 for a given snr. The 
degrees of freedonfl of the IC © are then defined as 



DoF(H) := limsup 



C(H;snr) 



■ log snr 

We call DoF(H)/M the normalized DoF. 



(3) 



III. Renyi Information Dimension 

It was recognized in jl] that information dimension is a 
suitable tool for characterizing the DoF in constant single- 
antenna ICs. The main conceptual contribution of the present 
paper is to show that information dimension is a natural tool 
for analyzing the DoF in vector ICs as well. This eventually 
leads to a unified framework for real interference alignment 
results as in ||5], f6|, that hinge on intricate properties of input 
distributions (singularity) and channel coefficients (number- 
theoretic), and "classical" interference alignment results, that 
rely on channel variations and vector signaling 1 3 1 . 

We start by collecting basic properties of information di- 
mension used in the remainder of the paper. 

Definition 1: Let X be a random vector. We define the 
lower and upper information dimension of X as 

mx)k) 



d{X) :=\imM^W}l 

fc^oo log k 



and dfx) := limsup ■ 

fe^oo log k 



respectively. If d{X) = d{X), then we set d{X) d{X) ^ 
d{X) and call d{X) the information dimension of X. 

Lemma 1 (^,Sd]ll)-' 1) Let X be a random vector in E". 

Then d{X),d{X) < cx) if and only if H{[X\) < oo. 

Moreover, in this case, we have 

d{X) ^ d{X) n. 

2) Let Xi, Xk be independent random vectors, such that 
d{Xi) exists for i — 1, ...,K. Then 



■Xi- 



K 



(4) 



d\ ■■ I = Y^d{x,). 

\Xk/ 

3) Let X and Y be independent random vectors in R". Then 

max{d(X), ^ d{X + Y). (5) 

4) Let X be a random vector in R" and F: M" ^ M" a 
linear isomorphism. Then 



d{FX) = d{X). 



(6) 



For a precise definition of C(H; snr) see [TT] Sect. 6.1]. 
"^For motivation on why to study this quantity see pi App. A]. 



The statements Q and ^ also hold for lower information 
dimension. 

Remark 1: An immediate consequence of (|4|i and (|6]l is the 
following. Suppose {vi,...,Vn) are linearly independent vec- 
tors in M" and Xi, ...,X„ are independent random variables 
with d{Xi) = 1, for all i. Then 



d{ Xivi 



XnVr 



ID 







(7) 



The nature of general probability distributions on M" can 
often be understood better by invoking the following decom- 
position result. 

Proposition 1 (e.g., /|72] Thm. 2.7.19, Ex. 2.9.14]): Every 
probability distribution /i on R" can be decomposed uniquely 
as 

where /^ac is an absolutely continuous, a discrete, and 
a singular probability distribution]! and a, l3,j ^ satisfy 
a + /3 + 7 = 1. 

Information dimension is typically difficult to compute 
analytically. However, for probability distributions that are 
discrete-continuous mixtures, there is a simple formula, which 
is an extension of (T., Thm. 3] to the vector case. 

Proposition 2: Let X be a random vector in M" with 
H{[X\) < oo and probability distribution /i that admits a 
decomposition fi = a/iac + (1 — a)lJ'd, where iiac is an ab- 
solutely continuous and a discrete probability distribution, 
and a £ [0, 1]. Then d{X) exists and is given by 



d(X) = na. 



(8) 



Another class of distributions admitting an explicit formula 
for information dimension is given by so called self-similar 
homogeneous distributions lfT4l . As in |[T], these special sin- 
gular distributions, suitably extended to the vector case, play a 
central role in the proof of our general DoF-formula for vector 
ICs. For a more detailed discussion of this matter we refer to 

na. 

IV. Main Result 
Theorem 1 (DoF-formula): For the channel ([T]l we have 



DoF(H)= sup dof(Xi,...,XK;H), (9) 



where 



dof(Xi,...,X;f;H) := 

^"Absolutely continuous" is to be understood with respect to Lebesgue 
measure. A discrete distribution is one that is supported on a countable set of 
points, whereas a singular distribution is concentrated on a set of Lebesgue 
measure zero and, in addition, does not have any point masses. An example 
of a singular distribution on R is the Cantor distribution |13 Ex. 1.2.4]. 



and where the supremum is taken over all independent 
Xi, ...jXk such that H{[Xi\) < oo, for all i, and such that 
all appearing information dimensions exist@ 

The proof of Theorem [T] as the proof of the corresponding 
result for the constant single-antenna IC in fT], builds on two 
results, whose extensions to the vector case are provided next. 
The first one relates information dimension to the high-snr 
asymptotics of mutual information in additive noise vector 
channels. 

Theorem 2: Let X and W be independent random vectors 
such that W has an absolutely continuous distribution with 

h(W) > -oo and H{[W\) < oo. Then 



lim sup 



I{X; y/smX + W) 



log snr 



diX). 



Remark 2: It is not hard to show that E[X^X] < oo im- 
plies H{[X\) < oo j9J. Consequently, finite second moment 
of X and W and h{W) > —oo is sufficient for the conditions 
in Theorem 12] to be satisfied. 

For the scalar case, Theorem|2]was proved in [8 Thm. 2.7], 
under the slightly stronger assumptions h{W) > —oo, E[l + 
\W\] < oo,E[l + \X\] < oo. Wu ES] Sect. 2.7] extended 
the result in |8, Thm. 2.7] to scalar W with finite non- 
Gaussianness, also allowing = oo. Particularizing 

Theorem |2] above to the scalar case also covers certain noise 
distributions with infinite non-Gaussianness, e.g., W can have 
a Cauchy distribution. 

The second result we build on is the following multi- 
letter characterization of the vector IC sum-capacity, which 
is obtained by extending 1161 Sect. 2: Lem. 1] to continuous 
vector alphabets with an average power constraint. 

Proposition 3: The sum-capacity of the channel (HJ is 



C(H;snr) = lim — 



K 



sup _ j:/(Xf;Yf) 

=1 



where X 



N 



X. 



(N) 



for random vectors X 



(n) 



S$ 1, 



for all i. 



and the supremum is taken over all independent 
Xf ,...,X^ satisfying 

m— 1 n— 1 

V. Implications of the DoF-Formula 

We now discuss implications of Theorem [T] 

Proposition 4: Suppose there is a permutation a of 

{1,...,K} such that a{i) ^ i and dct H£r(i)_,; ^ 0, for all 

i. Then we have 



DoF(H) K 



(10) 



PropositionHl which is the extension of [ 1 Thm. 5] to vector 
ICs, implies that for almost all channel matrices no more than 
K/2 normalized DoF can be achieved. 

*It turns out that it is not necessary to restrict the supremum to inputs 
satisfying the power constraint |9|. 



A. Singular distributions 

It was observed in IT] that input distributions with a singular 
component play a central role in achieving full DoF, i.e., 
the supremum in (|9]l, for the constant single-antenna IC. The 
natural extension of this result to the vector case holds, albeit 
with vastly different consequences as we shall see next. 

Proposition 5: Suppose Xi,...,Xk are independent ran- 
dom vectors in R*^, whose probability distributions are 
discrete-continuous mixtures. If detHi.j 7^ 0, for all then 



dof{Xi 



M 



s; 1. 



(11) 



The condition detH,;.j 7^ 0, for all which is stronger 
than the condition of Proposition |4] is satisfied for almost all 
H. Proposition |5] has far-reaching consequences as it says that 
restricting the transmit vectors to have distributions that do not 
exhibit a singular component, we can achieve no more than 
one normalized DoF. Singularity for scalar transmit symbols, 
as mandated in the constant single-antenna case, is impossible 
to realize in practice. In the case of vector ICs, however, a 
striking new feature appears. Singularity of the transmit vec- 
tors can be realized by simply choosing input distributions that 
are (continuously) supported on lower-dimensional subspaces 
of the ambient signal space. The following example illustrates 
this point. 

Example 1: Let K = 3, M = 2, and take 
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Choose the transmit vectors as 



Xi := Xi 



X2 X2 1 2 



(12) 



where Xi , X2 , X3 are independent random variables with 
absolutely continuous distributions and H{[Xi\) < 00, for 
i = 1,2,3. Note that the vectors Xi, X2, X3 are continu- 
ously distributed on lines in 2-dimensional space and hence 
have probability distributions supported on sets of Lebesgue 
measure zero; this renders the distributions singular Therefore 
dof(Xi, X2, X3; H)/2 is not bound by (fTTT i. Indeed, a simple 
calculation reveals that 



dof(Xi,X2,X3;H) 
'1 



d{ Xi 



+ di X2 



d X, 



{X2 + X3] 
{Xi+Xs] 
{Xi + 2X, 



Li -I- Z,^2) 

1 + 2-1 = 3, 



di^{X2+X3) 

d(^{X,+X3) 
-d({Xi+2X2 



and hence DoF(H)/2 ^ 3/2. Here, we used (Hi, ©, and 
(O. Since, by Proposition IH DoF(H)/2 < 3/2, the transmit 
vectors in ( fT2] i achieve the supremum in (|9]l, and hence full 
DoF. We emphasize that while the distributions of the transmit 
vectors in this example are singular, their marginals are non- 
singular; what is more, the marginals can be taken to be 
absolutely continuous, e.g., to have a Gaussian distribution. 
The reader can readily verify that this example follows the 
idea of "classical" interference alignment, as put forward in 

El. 

B. Parallel interference channels 

The system model (|2]) contains parallel ICs as a special 
case, obtained when j = diag(/ii [1], /i^ [A/]), for all 
For simplicity of exposition, we introduce the notation 

//ii,i[m] ••• hi^K[m]\ 

H[m] := ; ■■. : I m = 1, ...,M. 

\hK,i[m] ■■■ hK,K[m]/ 

Here H[m] is the interference matrix of the m-th subchannel, 
which is a constant single-antenna IC. 

It was shown in [2 1 that there exist non-separable parallel 
ICs, i.e., parallel ICs whose capacity region is strictly larger 
than the (Minkowski) sum of the capacity regions of the 
individual subchannels. This phenomenon stands in contrast 
to the Gaussian multiple access and the Gaussian broadcast 
channel, both of which are separable |2|. In this paper, we 
deal with the (weaker) notion of separability in the sense of 
DoF, i.e., the question of whether full DoF can be achieved 
by transmit vectors that have independent components across 
subchannels. 

Proposition 6: For parallel ICs, we have 

M 

DoF(H) ^ J2 DoF(HH)- (13) 

m— 1 

There exist examples where the inequality in ( fT3b is strict. 

Proof: The inequality in ( fT3] ) follows by restricting the 
supremization Q to inputs Xi with independent compo- 
nents Xi[l], -'f^-^^]' such that all information dimensions 
in dof (Xi [m], Xj<-[to]; H[m]) exist, for m — 1,...,M. 
Evaluating dof (Xi, Xk; H) for this class of inputs yields 



dof(Xi 



K 



..,Xk;'H.) 




\i:U^^AM]XAM]J 



K 

^h,^j[m]Xj[m] 



— d hi_j [m\Xj [m\ 



dof(XiH,-,^KH;HH), 



which proves ( fTSl ). A class of examples that renders the 
inequality in ( fTSl l strict, and is different from the example 



reported in ||2l, is obtained as follows. Take K 
with 



3 and M 



U[m] 




1,2, 



where A[l], A[2] are nonzero, rational, and satisfy A[l] ^ X[2]. 
Using ill Cor. 1], we get DoF(HH) < 3/2, for to = 1,2. 
Now, consider the transmit vectors 



X2 X2 



X3 :— X3 



where Xi , X2 , X3 are independent random variables with 
d{Xi) — 1, for i = 1,2,3. A calculation similar to the 
one in Example [1] reveals that dof (Xi, X2, X3; H) = 3, and 
therefor^ 



DoF(H) > 3, 



(14) 



which proves that DoF(H) > DoF(H[l]) + DoF(H[2]). ■ 
For parallel ICs that satisfy the condition of Proposition |4l 
e.g., when all subchannel matrices are fully connectedly we 
find through application of Proposition |6] that 

DoF(H[l]) + . . . + DoF(H[M]) GD DoF(H) M K 

Since we know that DoF(H[m]) — K/2 for almost all H[to] 
E Thm. 1], it follows from O that DoF(H)/Af = K/2 for 
almost all parallel ICs. Moreover, the left-hand side of ( fTsT i 
is achieved by transmit vectors with independent components 
across subchannels, i.e., joint coding across subchannels is not 
needed and therefore almost all parallel ICs are separable. The 
downside is, however, that by Proposition |5] the distributions 
of the scalar components of these transmit vectors are required 
to have a singular component. On the other hand, if we allow 
joint coding across subchannels, i.e., the use of dependent 
symbols across subchannels for the individual transmit vectors, 
full DoF can be achieved without having these symbols 
contain singular components, as the example in the proof of 
Proposition |6] demonstrates. We illustrate our findings through 
an example. 

Example 2: We consider a parallel IC with if = 3, M = 
2, and fully connected H[l] and H[2]. The fully connected 
constant single-antenna IC (i.e., M — 1) was studied in [5J 
where it was shown that K/2 = 3/2 DoF are achievable for 
almost all channel matrices. However, the result in fS] is not 
constructive in the sense of providing explicit conditions on 
the channel matrix for the underlying IC to have K/2 = 3/2 
DoF. In the following, we provide such explicit conditions on 
H[l] and H[2]. We also identify conditions on H[l] and H[2] 
for the underlying IC to have strictly less than 3/2 normalized 
DoF. We start by noting that DoF(H) is invariant to scaling 

'a more careful analysis shows that, in fact, we have equality in (14). 
*We call a matrix fully connected when all its entries are nonzero. 



of rows and columns of H[m], for any to [9] ■ It therefore 
suffices to consider the channel matrices 




TO = 1,2, 



(16) 



for TO = 1,2 (called the "standard 3-user IC matrix" in 
L5, Sec. VI-B]). Since H[l] and H[2] are fully connected, 
detHi j 7^ 0, for all and thus, by Proposition |4] we have 
DoF(H)/2 < 3/2. 

1) If d[m] is rational and a[TO], ^[to], c[to] are irrational, for 
TO = 1,2, then DoF(H[l]) = DoF(H[2]) = 3/2 by ^ 
Thm. 6]. Using Proposition |6] it then follows that 



DoF(H) 



Here full DoF are achieved by transmit vectors that have 
independent (across subchannels) scalar components. 
2) If d[l] = d[2] and a[l] ^ a[2], b[l] ^ b[2], c[l] ^ c[2], 
choosing each transmit vector to be continuously dis 

tributed along ^ 



a calculation similar to the one in 

3, and 



Example [T] reveals that dof{Xi, X2, X^; H) 
hence 



DoF(H) 



This is achieved by joint coding, i.e., by using transmit 
vectors with scalar components that are dependent across 
subchannels. 

3) If one of the entries on the main diagonal in ( fTSI l is 
rational and identical across the two subchannels, then 



DoF(H) 



3 

<2- 



This follows from an extension of H] Thm. 8], see (91 . 

C. The complex case 

The results stated so far apply to real signals and channel 
matrices. We next outline the extension of our main result. 
Theorem [T] to the complex case, thereby also providing an 
extension of the main result in 1 1 1 to complex alphabets. 

For a vector IC with transmit and receive signals in C^^ 
and G (^mxm ^ simply stack the real and imaginary parts 
of the transmit, receive, and noise vectors in ([T]), and stack the 
real and imaginary parts of the matrices j correspondingly. 
Then, apply the DoF-formula ^ to the resulting vector IC in 
M?^^ . Defining information dimension of a complex random 
vector accordingly as the information dimension of the vector 
obtained by stacking real and imaginary parts, we find that 
Theorem [T] applies directly to the complex case. 
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